User-aware Video Coding Based on Semantic Video Understanding and Enhancing

Traditional video coding is devoted to represent the video data compactly by dealing with low-level features (e.g., color, motion, texture, etc.) of the video. However, with the insatiable demand of Internet and increased use of multimedia, the bit-rate control issues are more and more important. In order to achieve more efficient representation for coping with diverse network or devices, many researches devised scalable video coding schemes which adaptively change the bit-rate according to the available bandwidth or user requirements. However, most scalable video coding algorithms only consider low-level features of video content in frame-based format without utilizing semantic information, which lose the possibility of improving coding efficiency by employing semantic meaning of content. Therefore, it is valuable to investigate methodologies of semantic-level video coding to produce more compact and flexible coding results for various user preferences. Semantic analysis for video content will provide richer information about the content and then assist achieving higher compression rate with good visual quality. Besides the coding efficiency, various functions are required in current video services, such as manipulating, searching and interacting with semantic-level objects. To enhance the flexibility and interactivity for accessing and manipulating the video content adaptively for different users, user-aware functionalities based on semantic video analysis should be discussed. In this chapter, we will discuss the theory and practice of user-aware semantic video coding, focusing on the aspect of semantic manipulation and user adaptation of video, including the semantic analysis techniques, scalable coding, user attention model construction, and useraware video coding, by considering requirements for different applications and giving an explanation about the methodologies for some example applications.